DEEP PRE-TRAINED LANGUAGE MODELS

Copied to clipboard

Challenge:	Existing pre-trained language models learn contextualized representations by using unlabeled text data and obtain state of the art results on a multitude of NLP tasks.
Approach:	They propose a pre-trained BERT model for Romanian language processing and compare it with multi-lingual models on seven Romanian specific NLP tasks.
Outcome:	The proposed model outperforms multi-lingual models on seven Romanian specific NLP tasks on sentiment analysis, dialect and cross-dialect topic identification, and diacritics restoration.

Papers with Deep pre-trained language models